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(57) Abstract 

A method and apparatus for real-time high-speed inspection of objects (Figures 1-2, 7a and 10a) involving storing 
digital signal mask information (Recognition Memory) of optical scans of objects at different magnifications (Figures 3-5, 
8 and 9), but with substantially the same field of view, and comparing digital mask information obtained by run scans of 
objects-to-be-inspected, at different magnifications, with the stored mask information to identify known or unknown por- 
tions of the objects, with adaptability for rapid 'teaching' of large sets of objects for storage and subsequent companson 

imtli r*»ol nKiArtc iinrfp.r insriertinn. 
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METHOD OF AND APPARATUS FOR 
REAL-TIME HIGH-SPEED INSPECTION OF OBJECTS 
FOR IDENTIFYING OR RECOGNIZING KNOWN AND 
UNKNOWN PORTIONS THEREOF, INCLUDING DEFECTS 

AND THE LIKE 

The present Invention relates to methods 
of and apparatus for real-time high-speed inspection 
of objects for such purposes as identifying known 
and unknovm portions thereof, pattern recognition, 
defect identification,, and the like. 

The art is replete with various types of 
pattern recognition and defect-monitoring schemes 
tending toward a fully automatic inspection system, 
such as those described, for. example, in the Automatix 
Bulletin (Burlington, Massachusetts) 1981 , describing 
Autovision II programmable visual inspection system, 
wherein programmed information used in a computer 
identifies object areas, perimeters, major and minor 
axes, holes. Such systems, however, cannot identify 
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defects that are <in>9ii 

^ are sn,aU perturbations in the 
measured obleet ar^ri 

sortvare control Fa^K 

• "^^ Pl^^Mre feature is 

measured and the image stored 1„ 

a.<.i...e„.,erated.,on.:i n^:"^"''^"" 
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Picture exe^ent is scanned. „ith storage oTx 

«-l-.es o. the o.,ect oni, that are o/i 

0«>er suggested approaches in.oive th se oT"'"^'- 

" - - —IC. opticax printed circuTt in ' ' 

system", by R r p ^''''"^^ 1-nspectlon 

- . t>y R. C. Restrick, SPLE Vol c . - 

Imaging Devices 1977 (p 7. , ' ^"'"'^ 

^P' /6-81), vhereln ^ , . 

oeen used to charar^t-^ • " "^^^ 

cnaracterize defect.? i« 

iects in scanning inspection 
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systems with precalculated minimum line width 
criterea. The present invention does not require 
such pre-calculation or programming and looks at 
general shapes broadly and flexibly, other mask 
systems ("A System For Automatic Visual Inspection of 
Printed Circuit Boards".' D. Antonsson et al, Linkoping 
University Publication. 1979.01.16) look for spacing 
between lines and holes, also with pre-calculated and 
hard-wired and programmed criterea and also without 
flexibility for new and general problems and appli- 
cations. In "A Process for Detecting Defects in . " 
Complicated Patterns" by Maskazu et al. Computer 
Graphics and Image Processing. 1973, Vol. 2 pp. 326-39, 
a different type of system is used that looks for 

small defects by enlarging and then contracting-, the | 
image-again concepts not required in accordance with 
. the present invention. The philosophical approaches 
underlying such and related techniques and the re- 
sulting complicated equipments and computer or 
microprocessor control programs operating in accor- 
dance with the same, have heretofore forbid a simple | 
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and universal approach to real-time high-speed 
object identification with facility for appliance 
to a wide variety of identification applications 
ranging from general pattern recognition to defect 
inspection and object or object portion identifi- 
cation for robotic control and the like. 

An object of the present invention » 
accordingly, is .to provide a new and improved method 
of and apparatus for real-time high-speed inspection 
of objects that are not subject to the above or ocher 
limitations and provide a universal approach to 
pattern recognition, defect identification and re- 
lated problems. 

A further object is to provide a novel 
high-speed automatic recognition and identification 
method and apparatus of more general utility, as well. 

Other and further objects will be explained 
hereinafter, being more particularly delinea-ied in. 
the appended claims. 

In summary, from one of its impoi -mt 
aspects, the invention embraces a method o' ceal-time 
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high-speed inspection of objects with the aid of 
image sensors and the like, that comprises, generat- 
ing signal images in the form of pixels of an object 
with such sensors at one or more magnifications i 
moving the object in a direction past the sensori 
quantizing the pixels of the images into a minimum 
number of light levels or colors required 'to character 
- ize.the object i storing the quantized pixels to create 
a two-dimensional stored image equal to the largest ■ 
desired field of viewi selecting predetermined desired 
fields of view and magnification from said two- 
dimensional stored image; dividing each such selected 
field of view into elements j setting the bright- • 
ness value of each element substantially equal to 
the average brightness value of the pixels contained 
within the element i quantizing each such element 
brightness value into a minimum number of threshold . 
levels required to recognize patterns of interest r 
applying each quantized pixel group at each magni- 
fication as a pattern of elements to a corresponding 
recognition memory, with each pattern of elements 
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serving as an address to said .e.ory; entering infor^ 
nation into accessed .emory address locations in order 
to teach the properties of kno.^ objects and character- 
istics thereof; and thereafter reco«mi.^ 

i^scognizing previously 

taught information as to objects anrt -v, ^ 

wujects and uheir characterls- 

tics by reading Information In" aach .emory location 
accessed by a pattern of ele:,e„t. and using each such 
Information to characterize the object. 

From another viewpoint, the Invention in- 
volves a method Of real-time .lnsj,e=tlon of objects at 
. predetermined region, comprising, optically scanning 
an area of Interest of the object at- said region, 
digitizing the scanned signals in real time to produce 
successive trains of digital signals corresponding to 
the optical scanned Unas, delaying the successive 

rrams of digital signals to produce a vertical raster 
-<=«-^- ="ins of digital signals corresponding 

to previously optically scanned Unas, reducing 



the vertical raster to a suVraster containing a 
predetermined number of substantially horizontal 
(H) and vertical (V) pixels i extracting from the 
sub-raster group? of pixels h by v in spatial distri- 
bution such that v^^V and hj^^H, where the sub? 
script i represent? the specific elements ranging 
from an element one pixel in size to an element V 
by B pixels in size? selecting an element thresh- 
hold value for each element size by predetermining 
a minimum and maximum number of pixels such that 
if the number of pixels contained in an element 
lies" within sai<3 minimun- and maximum values the ele- 
ment value is set to the value 0 orl» with such 
thresholds varying with the size of the element » 
delaying each element by N, 2N, 3N, etc. horizontal 
pixels and by N, 2N, -SN, etc. vertical pixels to 
generate a mask representing a visual pattern 
wherein each element of the mask is displaced a. 
full element from the previous or conttguous hori- • 
zontal and vertical element;, in a teaching mode, optically 
scanning a known object to generate a mask as above, serving 



as « Address, and s.or,r^ a code number at each 
a«re== to teach the recognition of the ' 
■ desued Object or pa«e™, «hen operatins with an 

object to be insoected ir, „ 

nspected In run node, reading the 

code number to determine tf the ™ae. generated 
during the optical acannlng of the objeot-to-be- 
inspected corresponds thereto. Indicating the recog- 
nation Of the desired object, or does not corres- 
pond thereto; and indicating ir the object or 
pattern has not been previously seen. 

Fro. stiU a „ore general, viewpoint, the 
invention also embraces a method of real-ti„e high- 
speed Inspection of Objects vith the aid of inage- 
senslng scanning cameras and the Xl.e that co»- 
PrUes. storing digital signal „ask information 
corresponding to video images of regions of pre- 

determined fields of view of v' 

View Of a known object at 

effectively different maenifi r^^i-^ 

•^magnifications, successively 
scanning contiguous regions of an object with such a 
camera, for each of said predetermined fields of vie.. 
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and digitizing the image scans to generate digital |. 

signal mask information; repeating such scanning | 

at effectively greater magnification with the same f 

field of view to generate magnified digital signal I 

mask information; comparing the generated digital I 

mask information at different magnifications with t 

. . • i 

the stored digital mask information to identify I 

known or unknown portions of the object; and indi- I- 

eating the identification of such known or unknown 

portions. Other features of the invention and pre- • 

ferred and best mode embodiments, including pre-- ' 

ferred apparatus and constructional details, are 

hereinafter presented. 

The invention will now be described with t 

reference to the accompanying drawings. Fig. i of | 

which is an explanatory diagram illustrating orienta- I 

tion problems in inspection systems; I 

Figs. 2,3, 7 and 8 are plan views of |: 

illustrative defects in one of the important areas \ 

of application of the invention to printed circuit f 

board inspection and the like; f, 



Figs. 4 and 5 are *4n.<i 
views o-F similar diagramatic 

vievs of systems for im=«,- 

ior imaging at differ«>««. 
cations I °J-"erent magnifi- 

^5-8- 6 is a block diaeran in 
--ry apparatus useful in ins '"'"^^^^^^^^ 

const_edin.aeeordaner.i:^^^^^^^ 
P--30P., Of t.e in.ention "^^"^"^ 
Fig. 9 contains views stm^i 

f^S- 10 ta a similar view 111, . 
-Wftlne positions of "l"«"tl„s 

ot memory maskf 

« -Pec«o„ apparaJ: opLa" T^" 

Invention, ^ . "cordance vlth 

iined in the m«« . * -uiai. ©ut- 

^ ''^'"^"S Chart Of Fig. 14, 
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Flgs. 15 and 16 are schematic block 
diagrams of scanned element quantization and delay 
registration in accordance with the invention* 

Figs. 17 and 18 schematically illustrate 
successive scanned bands of pixel groups and the 
quantizing of the samej 

Fig. 19 is an alternate arrangement to 
Figs. 17 and 18 r 

Fig, 20 is a more detailed diagram of the 
. scanning window memory used in the system of Figs. 
11, 13 and 16 i . 

Fig, 21 is a block diagram of the thres- 
hold comparators useful in the embodiment of Fig. 16 r 
Fig. 22 is a similar diagram of a preferred 
prototype implementation of the invention j and 

Figs. 23(a)- (e) are block diagrams of 
parts of the inspection and correlating pattern in- 
formation processer of Fig, 22. 

Underlying the invention are certain pro- 
cedural or philosophical approaches that are quite 
distinct from prior techniques and that are helpful 
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to review for an xmderstandtng of the invention i 
For illustrative purposes, the problem of recogniz- 
ing or identifying defects in an object, such as a 
printed circuit card or the like, will be referenced? 
it being understood, however, that the invention is 
clearly applicable to pattern or object identification 
in the general sense, as well. Such typical defects 
in, for example, printed circuit cards and the like 
are printed conductor breaks, pin holes, a neck or 
thinness in a. conductor, and short circuits and the 
like. Many of these defects are microscopic in size. 

The probleni in this illustration may be to 
locate on the printed circuit card various size de- 
fects, including a defect that may require a fair 
degree of magnification before one can visually see 
the defect. From the heuristic standpoint, an 
observer attempting visual inspection may first 
scan the complete area, looking ovisr small areas 
thereof to see if relatively large defects can be . . 
found, but without viewing the entire cax'd in one 
glance I rather, concentrating on these smaller areas. 



section by section, and inspecting contiguous regions. 
To look for smaller defects* the observer may resort 
to the use of a magnifying glass. The field size 
as viewed by the eye will be the same as that viewed 
without the assistance of the magnifying glass i but 
now the observer will actually be looking at much . 
smaller regions that have been magnified to some degree 
With this greater magnification, inspection of con- 
•tiguous regions will enable, viewing any small defects, 
breaks, etc. The. next step may be to inspect even 
smaller areas to identify even smaller defects, per^ 
haps very fine . pinholes , requiring an even more 
powerful magnifying glass for inspecting contiguous 
regions of interest, as before. The concept that in 
each case the field of view presented to the eye 
remains constant, but the amount of magnification 
and actual distances on the -object have been blown 
up to fill the visual field of view, is an important 
concept within the context of the invention in pre- 
ferred form. 
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•Constder that a small field of view Is 
"ade up Of uttle picture elements, (pixel,), i^ese 
Ptxels are grouped to create elements of binary 
value 1 or 0 and the elements are then grouped to 
create a ™ask. In thla application, a ™ask 4x4' 
elements Is created which can easliy be mapped Into 
a 65 K dynamic or static memory. The technique for 
effectively varying the n^agalflcatlon to be provided 
to this mask resides In the use of picture point 
(or pixel) averaslng. somewhat analagously to human 
viewing techniques. If one holds something very close, 
a lot Of the details are apparent,' but as one looks at 
an Object farther away, one tends to average the fine 
Picture points Into one. In accordance with the ' 
Invention, somewhat similarly, elements are formed 
•>7 producing a kind of averaging or weighting func 
tlon on a small group of picture elements or pixels 
»tth threshold averaged values selected such that 
the element value. 1 or 0, represents the averaged 
Pixel brightness within the element area, for example. 
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if an element is to be represented by a group of 
four pixels. 2 horizontal (h) by 2 vertical (v) , 
it may be decided that if three of the four pixels 
are white (with common binary digits in the mask), 
the whole block will be considered as of white 
brightness t or if three are black, this may be the 
threshold for treating the block as black. | 

Thus, the threshold is set on a summing 
. operation, and in real time, the final mask composed 
in this example of 4 by 4 elements is finalized with 
an averaging or weighted function of the pixels- - 
such architecture lending itself to extremely high 
speed implementation. At a 5 megahertz data rate, 
as an illustration, the entire function composed of 
first forming all of the 16 elements and then de- 
ciding whether it is a "good" mask or a "defect" 
mask can be achieved. That determination can be 
made by storing "good" masks and comparing a real- 
time mask scan with the "good" masks to id- atify 
a defect. Alternatively, design masks for all 
defects may be stored, whereby mask matching would 
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identify the defect. Since, moreover, the inven- 
tion embraces looking at local features (perhaps a 
break or a pinhole) , it is possible to use many 
inspection- units simultaneously looking at the 
field in parallel. This may be somewhat analagous. 
from the human standpoint, to having several ob- 
servers, each having a magnifying glass, sinjuL 
taneously moving their magnifying glasses over 
different sections of the. objects to be inspected. 
Because the field of view is adjusted such that a 
defect will fit inside the same, parallel prccessing 
units of this type could be employed, contiguous 
with one another, or overlapping and enabling in- 
spections of very large fields in real time at very 
high speeds. 

It. is believed conducive to an understand- 
ing of the invention to treat first with the general 
philosophy underlying the inspection process thereof, 
involving certain frequent operations for recognizing 
objects and inspecting them for imperfections and/or 
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characteristic identification marks in order to 
provide the sequencing framework for constructing 
an automated apparatus capable of performing such 
tasks at extremely high operating rates. 

The methodology or architecture for im- 
plimenting such an apparatus from a systems stand- 
point will then be described in terms of a series 
of functions which can easily be performed by 
machine. The mdchine of the invention itself will 
then be delineated to illustrate exemplary imple- 
mentation for performing the required functions in 
an efficient manner at very high operating rates, 
analyzing the entire object in real time such that 
this type of system is particularly well suited for 
production line inspection applications. 

Turning first to the inspection philosophy 
underlying the invention, it is useful to consider 
the general ways in which a set of objects may differ . 
from one another, such as differences in imprinted 
data or pattern, or the presence of a defect or other 
distinct characteristic. From an inspection standpoint 
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the same algorithm can be used to locate the pattern 
or defect that uniquely defines the object. For 
illustrative purposes i and because such is an im- 
portant current application of interest, let us 
consider the use of the invention to inspect sets 
of printed circuit cards to detect imperfections 
such .as cracks, holes or other .surface irregularities. 
One may begin by first examining a single card at 
the highest optical magnification available, care- 
fully looking for any noticeable faults while examin- 
ing contiguous or overlapping fields of view (FOV) 
until a defect (or, in the more general case, a 
pattern of interest) is detected. in one of the fields 
A most desirable feature of this recognition pro- 
cedure, in accordance with the present Invention, 
is to enable operation of the same with decx'eased 
sensitivity to angular rotation of the card or other 
object under inspection. Consider, for exa:::ple, a 
printed circuit card PC that is, say, twelve inches 
long and six Inches wide and that has a pinhole 
defect A with diameter 1/1000 inch (1 mil) located 
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in the upper left corner, as identified in Fig, 1. 
If the PC card is tilted at an angle 9, say one 
degree (illustrated in exaggeration in Fig. 1), 
this represents less than 0.3% of a circle, and 
such angular component of rotation would not be 
noticeable to a human inspector or observer. The 
linear component of displacement S, however, is 
given by S = Re (in radians) > such that for the 
illustrative R of 12000. mils, S will be 200 mils, 
which is 200 times greater than the size of the 
hole A in this example. It is for this reason 
that prior art pattern matching recognition algorithms 
are not suitable for inspecting patterns that are 
small relative to the size of the object? though a 
human observer does not have this problem and would 
simply locate the defect at its rotated cpordinates. 

In human observer recognition, the person 
analyzes the contents of each field independently; 
and this quality is a feature that is also incorporated 
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Into the inspection philosophy of the present In- ■ 
ventlon. Thus, a first operation that the Inspec 

tlon technique and apparatus of the Invention performs 
is tot 



1. Examine the contents of each field 
of view lndeDenrt»n.-i^,, searching for a 
pattern of Interest that lies within the 
' field. 

The pinhole type defect A Illustrated In Fig. 1 Is 
again shown at A In the lower right l„ Fig. z. which 
Illustrates also another type of frequent printed 
circuit card defect, namely a short-circuit metal connec 
tlon B between a conductor 1 on the printed circuit 
card PC and a bonding pad. so- labelled, connected 
with a further conductor 2. Pig. 2 Indicates that 
the pinhole and short defects A and B are to be ob- 
served both at relatively high and at relatively low 
magnification, as more ' particularly shown In Fig 3 
Portions A and B of Fig. 3 represent the appearance 
Of the respective pinhole and short defects at high 
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ma^ification (and as compared with a normal con- 
ductor edge at C), while portions D and E represent 
the same respective defects at low magnification. 

Certain larger defects cannot readily be observed • 
within a given field of view at such high magnifi- 
cation » and it is thus frequently necessary to re- 
examine the object at lower magnifications. Con- 
sider, for example, the short defect B of Figs, -2 
and 3, in which the bonding pad has been shorted to 
conductor 1, tangent to the pad. With the high 
magnification field of view B, the short Is 
indistinguishable from field C which shows a normal 
section of nietalization joining conductor 2 to the 
pad. Iff however, the magnification is reduced such 
that the entire short lies within the FOV, Fig. 3Dt 
recognition of the short as such is greatly facilitated. 
Fig, 3E illustrates, on the other hand, that this low 
magnif icationt so suitable for large, defect identi- 
fication is not suitable to search for smaller defects 
such as the pinhole A, In the more general, case, one 




nsay desire thus to examine an object at a variety 
of different inagnifications. Thus, the automated 
inspection system of the Invention will alsoi 
2.. Examine the entire object at a 
variety of different magnifications, 
analyzing each FOV as described in 
operation 1, above. 

In other applications, it is advantageous 
to combine pattern information obtained at various 
magnifications to identify defects or other dis- 
tinguishing characteristics. An illustration would 
be the case in which one must locate a break, for 
example, between a conductor and a bonding pad. One 
may first locate breaks at high magnification and 
then, without moving the circuit card, decrease the 
magnification to- confirm presence of the pad and ad- 
joining conductor. In this mode, identification of 
the pad- break- conductor configuration is obtained by 
correlating pattern information obtained at two differ- 
ent magnifications. The machine of the invention, to 
perform this recognition task, will thusi 



3. Correlate pattern Information ob- 
tained at different magnifications of the 
same object area to identify patterns of 
interest. 

Unfortunately, there may exist certain 
large patterns or defects which will not lie within 
the FQV of the magnifying apparatus (such as a micro-, 
scope) even under the lowest magnification provided. 
When this occurs, the invention provides, for viewing 
contiguous or overlapping fields at this low magni- 
fication and constructing a complete pictvire equiva- 
lent to what would have been seen had a sufficiently 
low magnification objective been provided. For the 
machine of the invention to operate with this feature, 
it will, therefore I 

4. View contiguous or overlapping fields 
and combine pattern information from each 
field to identify large patterns of in- 
terest. 

It is important to recognize at this junc- 
ture, two distinct fields of view. One field is the 



^rea of the object viewed by, for example, the 
microscope objective. Ihe other Is the area of. 
the microscope eye piece viewed by the eye. in 
the case of a human observer. The field of the 
objective is a variable which is a function of 
magnification, whereas, the field of the human ' 
eye Is a. constant. It Is from this constant field 
that the brain obtains all Its Information, with 
the plcroscope serving to adjust the desired ob- 
ject image to flu the visual field. For this dls- 
«=usslon It is advantageous to model the eye as a 
sroup of discrete visual receptors, such that contl- 
suous image sections excite contiguous visual re- 
ceptors. The human recognition system, the brain, 
then processes the data provided by these receptors 
and attempts to make a decision as to object or 
pattern Identity. 

In accordance with the automated recognition 
syste. Of the invention, a so.ewhat similar .ode is 
adopted wherein the system will, 
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5. Divide each optical field at 
each magnification into a constant number 
of picture elements (N^) and present only 
these N elements to a processing unit for 
recognition of the viewed pattern. 

It should be noted » however i that the eye 
contains millions of receptors and the brain con- 
tains billions of memory locations To construct 
an apparatus of this complexity would i of course J 
be extremely difficult and costly^ A' general practi- 
cal system of this type of architecture, however » 
2 

in which N and the memory capacity has been reduced » 
is implemented in accordance with the invention, as 
later described • 

In the general case where each picture ele- 
ment (pixel) responds to L different levels of light . 

intensity or color, the total number of possible 

2 

patterns P that can be imaged onto N elements is ' 
given byt 



Consider, for example, illustrative numbers of 
L - 256 levels of intensity or color and » 16 
elements. The resulting P = 3.4 x 10^^ patterns is 
an absolutely enormous number to contend with. 
Fortunately though, when analyzing an image, the 
human recognition process is frequently precondi- 
tioned by a prior knowledge which ckn be used greatly 
to' reduce the number of possible patterns. For 
example, when looking for printed circuit board 
defects one concerns oneself solely with the ab- 
sence and presence of the conductive metalized 
traces. Variations in image intensity resulting 
from non-uniform illumination, light scatter and 
varying degrees of reflection due to oxide buildup 
and the like are totally Ignored by the human ob- 
server, and the image is- effectively quantized into 
two levels, one representing the presence of metal, 
and the other, representing its absence. In effect, 
thus, we have reduced L to two levels. Reconsider- 
ing the numbers of the above illustration, therefore, 
for L . 2 levels of intensity and = i6 elements. 
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the number P becomes 6.5 x 10^ patterns, a reduction | 
of P by a factor of 5 x 10^^. The machine of the in- | 
vention is thus constructed to adopt such efficient | 
considerations and it willt | 
6. Use the minimum number of light 

levels or colors required to represent 

the object of interest, 1 
From perhaps a more precise viewpoint, one 
actually wants to reduce the number of patterns P. 
If conditions exist where N can be reduced at the cost 
of increasing L, such that P - is thereby re- 
duced, then this may be a preferable mode of .efficiency. 

. In summary, therefore, the image acquisi- 
tion function underlying the invention consists of 
the following three operations i 

a. Obtaining images of the object at one 

or more magnifications i 

b. Constructing these images using the itiini- 

mura number of required light levels or 
colors I and 

c. Dividing each image at each magnification 

into discrete spatial elements. 



The image recognition function of the 
invention, following such image acquisition, con- 
sists of performing any number or combination of 
the following four operations to locate patterns 
of interest I 

a. Examining the elements from each field 

at each magnification independently, 
looking for a pattern of interest within 
each field? 

b. Correlating patterns of the. same object 

area generated at different magnifica- ' 
tions t 

c. Combining pattern information generated 

from contiguous or overlapping fields 
at a single magnification? and 

d. Performing recognition operation c at 

each of the different magnifications, 
and correlating the obtained pattern 
information. 
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It is now In order to examine the system 
methodology and functional description for imple- 
menting the invention. 

The first system function that must be 
implemented is the acquisition of object images at • 
different magnificationt The simplest and perhaps 
most conceptual means to achieve this goal consists 
of placing a two-dimensional teleyisiont CCDi 
thermal- sensing, or other suitable sensing camera at 
the eye-piece of a microscope. In this mode, the 
object will be repeatedly scanned at each of the 
required magnifications. Such technique, however, 
has the inherent disadvantage of being relatively slow, 
since objects are viewed at each magnification in a 
sequential Kanner, Thus, the image acquisition 
time is proportional to the number of different 
magnifications incorporated. In addition, if the 
object is moved at a rate faster than the ceadout 
time of the camera, the image will be bluii.ed and 
distorted. This is frequently observed or. home 
television sets, for example, when viewing a fast 
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moving object such as a baseball in flight. This 
blurring phenomenon is an additional factor which 
limits the maximum inspection rate. 

An alternative method, accordingly, is 
to obtain all images at each of the different magni- 
fications in parallel. This can be achieved by 
having a separate camera for each different magnifi- 
cation as shown in Fig. 4. wherein camera 1 effectively 
operates at 5 x magnification, camera 2 at 10 x. 
and camera 3 at 20 x, for example. The object, so- 
labelled, would then be moved past each of the 
cameras, as shown by the arrow, or .beam splitters 
and mirrbrs may be used, as in Fig. 5. to image the 
object onto each camera. This technique, however, 
has numerous disadvantages. It is costly and techni- 
cally complex, requiring as many cameras as there are 
different magnification tj^es, and critical optical 
alignment between cameras Is required to maintain 
spatial relationships between patterns obtained at 
different magnifications. In addition. Image 
blurring still occurs If the object Is moved too fast. 



1 
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A third method, and the one implemented in 



accordance with the preferred embodiment of the in- 
vention, utilizes a high resolution large field lens 
L^, Fig* 6, (such as, for example the type Photor 
produced by LeitZf or APO-EL-Nikor produced by Nikon) 
to image the object onto a single high resolution 
linear charge coupled device imager (CCD) (such as, 
for example t the type 1728 pixel CCD currently 
produced by Reticon,' or the type 2000 pixel CCD 
produced by Fair'child )f shown at CCD in Fig. 6, The 
output of the CCD is then transferred into a two-- 
dimensional buffer memory BM and then into a sliding 
window memory SWM to create large field high resolu- 
tion images. This technique enables the resolution 
to be equal to that obtained from high magnification 
objectives, while the FOV is equal to or greater 
than that obtained from low magnification objectives. 
In addition, object blurring can be totally eli- 
minated by moving the object perpendicularly to 
the axis of the CCD and reading out the CCD into 
the buffer memory BM each time the object image has moved 
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a distance equal to one CCD element, which .ay be. 
for example, approximately 1/2000 of an Inoh 
Specifically, the analog signal output of the CCD. 
after conversion to digital forn, by a conventional 
sampling pixel ,uantl«r, so labelled. Is trans- 
ferred In Fig. 6 Into the buffer .e»ory BH which 
can be modelled as a series of shift registers SR 
each equal In length X to the number of CCD ele- 
ments, ihe output (OUT) Of each shift register SR 

is connected to the input (IN) of rho 

Hui- vin^ or the next register 

and also to one row of the ?n hi— . ^ 

cne Sliding window memory 

SWM, as shown in Fig. g. 

Ihe image appearing m the window memory 
SWM is similar to what would be seen by moving a 
-snlfylng glass over the entire object, as sche- 
-tlcally illustrated in Fig. 7, wherein (a) illus- 
trates, the resolution and FOV seen In the window 
memory SWM as compared to what is seen at (b) and 
(=). by a conventional camera viewing the evep.e^. 
Of a microscope at high and low magnification, re- 
spectlvely. 
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In addition, the optical configuration 
incorporated in the system of Fig. 6 is extremely 
flexible. The image size of the object produced 
on the CCD can be magnified or demagnified (from 
1/3 X to 3 X using said Photor lens L^i 2 x to 
10 X or 1/2 X to 1/10 X using said APO-EL-Nikor 
lens) by simply moving the relative positions of 
the CCD, lens and object. Since the CCD is 
relatively very long (1728 pixels for said Reticon 
apparatus and 2000 pixels for said Faiirchild equip- 
ment) and each pixel is extremely small (0.64 mils 
for Reticon, 0.5 mils for Fairchild), a large 
variety of high resolution large field images are 
obtainable. 

It is not deemed practical to recognize . 
patterns from this large field high resolution 
(LFHR) image directly, because the number of possible 
patterns is astronomical. If the image were divided 
into 32 X 32 pixels, for example, each represented 
by two levels of light, there could be as many as 

,^308 ci^r-^ 

i possible patterns. Instead, in accor- 
dance with the invention, information is extracted 
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Of the .l„e.e« Specl«caU. 
the rouowms de^asnlneatlon p„ce.u„ U app^e. 
to the LFHS image to obtain t-K 

oi3tatn the appropriate K)Vs 
and resolution required rr. 

^ sxHUilate all desired 

magnifications in parallel, 

A. Select the FOV's rh=*. 

W s that represent the desired 
magnifications. 

An illustrative selecti-«« * 

selection is shown in Fie P 

ponding to Fig. JU) ^ ^' ®' 

S is. 7(a), for a high and low magnifi. 
cation objective. . 

B Biviae each K>v l„.o ele™e«.. 

r the „.pective hl«h ana ' 

low magnification fields of • 

nelds Of view of pig. g 

shown m Fig. 9 ^3 

- 16. at (a) and (.>. ^« 

"'^^^^-^ Of each element ' 

e^ual to the average brightness val.e 
Of the Pixels within the element. 



In mathematical terms t if i 

Ej^j - brightness value of the element in 
horizontal position i and vertical 
position Jt 

Pxy - brightness value of the pixel in hori* 
zontal position x and vertical position 
y$ and 

N « total number of pixels contained within 
each element, 



thent 



Ej^j « 1/N 




If the brightness value of each element is quantized 
into one of 256 possible light levels ^ and there are 
16 elements, then as many as 3.4 x 10^® (256)^^ 
possible patterns could exist. Fortunately, however, 
in most applications, one can represent each element 
by considerably fewer light levels. In the case of 
the printed circuit card PC, for example, as few 



brls^tnes, for .ach of 16 elements, nela, SS.536 
(2 ) POsslMe patten,,. i„ general, therefore. i„ 
«cordance wtth the Invention one would, 

D. Quantize the brightness value of each 
element Into the mtnlmu™ number of 
Xlsht levels required to represent 
patterns of interest. 

a specific Illustration, consider 
~ln. each .le.ent Into ^ levels Indicating 
the presence and absence of ™etal on the card PC. 
If. Jen. .ore than , given percent of the pl«U 
"Ithin an element Indicates the presence of „etal ' 
the entire element win Indicate the presence of ' 
"etal. In mathematical terms, let, 

Pxy - quantized brightness value of the 
Pixel m the Window memory at posi- 
tion X, yt 

- 1 If metal- Is present on the card and 
= O If the metal is absent on the card, 
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Ej^j « brightness value of element t, jt 
« 1 if the element contains raetal and 
= 0 if the element does not contain metal i and 

S « threshold value. 



Then, 



Pxy I and 

all X within all y within 
element E^^ element E^^ 

= Of otherwise. 

2 

This group of N elements forms the actual 
patterns which are to be recognized and therefore 
will be . referred to . as a pattern of elements (POE). 
Each POE is applied to a recognition memory for 
positive identification. Each different magnifi- 
cation (or FOV) has its own recognition memory. 
In general, if each element is quantized into L 
discrete light levels and each POE contains ele- 
ments, then there must exist L memory locations, 
at each magnification, to store pattern information 
about each of the L total possible patterns. 
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Since the mapping must be unique between^ 
each memory location and POE, the POE bit pattern 
Itself is used in the invention as the memory 
address, and information about each particular POE 
is stored at that address. 

The specific information is a function 
of the application and desired task. If that task ' 
is to locate defects on the printed circuit card as 
in the above illustration, and a defect is defined 
as any pattern not present, on a good card, the 
memories are programmed as follows. A good card 
is scanned, and at each magnification, a valid bit 
is entered at each memory location accessed by a 
POE. This mode of operation is referred to as the 
"learn mode" because patterns are "learned" by the 
memories. To check cards, the system is now placed 
in "inin.mode." Each card is scanned and all POE 
are applied to the appropriate recognition memories 
for positive confirmation. Any foreign POE not 
previously learned is classified as a defeot. 
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In other applications,, one may require 
that a minimum number of foreign POE be present 
within a given spatial region before a defect is 
said to be present. It is important to note that 
when, selecting the range of inspection magnifica- 
tions, one must ensure that the highest magnifi- 
cation does not respond to nor.mal surface varia- 
tions in textural composition* 

In still another application, one may* 
desire to teach the machine to recognize different 
objects. An example useful in robotic and many 
other applications, is to teach the machine (learn 
mode) to differentiate among* different tools, such 
as a plier, screwdriver, and hammer. One indepen- 
dently scans each of the three objects, assigning 
a different code number to each tool. If the plier 
is assigned code #1 and is scanned first, the 
number 1 is stored in each location accessed by 
a POE» If the screwdriver is scanned next and is 
assigned code #2, a number 2 is placed in each 
blank memory location accessed by a POE. If the 
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•location i, not blank, but contains the number 1, 
this Implies that the particular POE exists In bith 
tools and the number 4 Is stored In this location 
to indicate that the pattern is not u„t,^e. The 
hammer is now assigned #3 and scanned Into the sys- 
tem. All accessed blank locations are programmed 
with the number 3. All accessed non-blank locations 
are programmed with the number 4. m run mode, a 
tool is scanned, and each memory address accessed 
by a POE is read. The frequency at vhlch the 
numbers 1. 2 and 3 occur is computed. The object 
is identified by the number having the highest fre- 
quency of occurence that also exceeds some pre- 
determined miniinum value. 

• ^""""^ "«chlne has been taught 

an POE's Characterising an object. It Is essential ' 
that POE-s be calculated after each new column of ' 
Pixels is transferred from the CCD buffer memory 
BM Into the window memory SWM (Fig, 6). This can 

be understood with the aid of the fono„. 

ui une tollowlng example. 



Consider a U-shaped object with three 
critically located mounting holes as shown in Fig. 10(a), 
To obtain the best resolution using the minimum num- 
ber of elements f the field of view and element size 
shown in Fig. 10(b) is chosen. Notice, however, that 
one has been careful to place the object precisely 
in the center of the FpV (within the window memory) . 
In reality, this condition will only exist at a given 
instant of time as the CCD is scanning the object. 
At another instant, the image may appear as shown in 
Fig. 10(c) ,. shifted to the right. To insure that the 
machine recognizes the desired pattern of Fig. 10(b) 
and is also taught that the pattern of Fig. 10(c) may 
occur and is not a defect i POE's for both images 
must be taught to the machine. 

Another application may involve teaching the 
machine to recognize a certain pattern for which there 
is no physical example. To teach this pattern to the 
machine, the correct recognition memories are first 
accessed directly via a memory input/output port. 
Identification numbers are programmed into the 
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men,ory addresses thet would be accessed by the 
POE's representing this specific pattern. 

In run ™ode. the contents of aU accessed 
-»ory locations are examined. When the contents 
equal the programmed Identification numbers, the 
specific pattern has been located. 

In summary, the methodology' and machine 
functions necessary to Implement the inspection 
apparatus are presented In the block diagram of 
Fig. 11, involving the following flow of functions, 

a. Moving the object In a direction perpen- 

dicular to the axis Of the CCD or other 
imager t 

b. Quantizing pixels from the CCD output 

into the mininnam number of light levels 
or colors required to characterize the 
object Ca/D in Fig, n), 
o. Storing quantized pixels in a buffer to 
create a two-dimensional i.age equal to 
the largest desired field of vlev (BM 
in Fig. n), 
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Choosing desired fields of view (FOV) 
and magnifications from the two-dimen- 
sional image (in SWM, Fig. 11), 

Dividing each FOV into elements (block 
(e) of FOV PROCESSOR in Fig. 11), 

Setting the brightness value of each 
element equal to the average brightness 
value of the pixels contained within 
the element (see "AVG. BRIGHTNESS" output. 
Fig. 11), 

Quantizing each element brightness value 

Into the minimum number of levels required 

to recognize patterns of interest (block 

(g) of FOV PROCESSOR, Fig. 11), 
2 

Using each N quantized pixel group re- 
ferred to as a pattern of elements (POE) , 
as the address to a recognition memory 
("(h)", Fig. 11), 

Using a different recognition memory for 
each different FOV (magnification) (FOV 
PROCESSORS for field 1 and field 2, 
Fig. 11), 
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J. Teaching the machine to recognize an 

object (learn mode) by computing POE's 
after each new column of CCD pixels is 
transferred Into the sliding window ' 
n.emory and storing Information character- 
izing each POE into the memory location 
accessed by the POE ("(j)" m Fig. u), and 
k. Recognizing a previously taught object by 
computing FOE'S after each new column of 
CCD pixels is transferred Into the. sliding 
window memory, reading the information in 
each memory location accessed by a POE. 
and using this information to characterize 
the object C"(k)", Fig. li). 
While Fig. u is a block diagram that 
summarizes the system functions described above. ■ 
. to implement the final machine as in a practical 
high-speed, real-time, cost-effective system, these 
functions may be performed In a somewhat different 
order than indicated, as hereinafter explained. 
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Now turning to practical and preferred 
system implementation of the technique of the In- 
vention, It should be recognized that using the 
exact sequence of operations presented in Fig. 11 
is extremely. complex and costly though, if desired, 
it could be Implemented. If, for example, the FOV 
extracted from the window memory SWM contains 32 x 
32 (1024) pixels and the CCD Is operated at a- 5 
megahertz data rate (200 nanosecond period), all 
1024 pixels required to compute the POE for this 
field must be processed such that a new POE is 
generated each 200 nanoseconds (ns). Processing 
Involves . computing the average and quantized bright- 
ness values for each of the elements. In addi- 
tion, this entire computation must be performed 
for each different FOV simultaneously extracted 
from the window memory SWM. 

This complex operation can be greatly 
simplified by realizing that all the pixels re- 
quired to compute the first of the elements 
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(shovn « E^^ at (e) 1„ Fig, n ana »ore fully 
delineated In Fig. 12(a)). u transferred from 
the CCD buffer BM into the window .e^ory SWM prior 
to that required to compute' the foUowlns element 
(Et_2,. FiS. 12(a), etc. up to element E^). One ' 
can therefore compute' the element values in a 
temporally sequential manner, storing them until 
all N elements have been computed and thSn apply 
them to the address lines of the recognition 
memories. By doing this, the window memory SWM 
need only be of sufficient size to store the num- 
ber of pl«i, contained in the largest element, 
reducing the memory requirement of the window by 
.e factor of n2. Mother advantage of this techni- 
que is that only one pi«l averaging and quantizing 
unit is required to compute all element values, 
rather than units. Ihe last and perhaps most 
important feature of this technique is that all 
N element values required for each consecutive 
POE can be obtained in one clock cycle (200 ns) 
If the n2 element values are stored in a tapped 
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delay line memory. The details of implementing 

this architecture will nov be described. 

Consider a field of view containing 

2 

pixels that is divided into N elements as shown 
in Fig. 12(A) and at (e) in Fig. 11. Let the 
window memory be reduced in size such that it 
only contains those pixels being within a single 
element. This would correspond to a window memory 
capacity of (C/N) pixels, where it is assumed 
that (C/N) is an integer. The solid lines enclos- 
ing the circles in Fig. 12(B) are intended to 
represent the FOV of the window memory forming 
rectangle I of Fig. 12, with the pixels represented 
by the group of circles. The first group of (C/N)^ 
pixels that fills the window memory constitutes all 
the data required to compute element E^^^ of Fig. 12(A) # 

The rectangle I of pixels (circles) is also 
shown in enlarged form in Fig. 13, showing details 
of the brightness averaging and quantizing func- 
tions ' (e) and (g) of Fig. 11, and which is to bs 
now discussed in connection with successive times, 
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as indicated on the tin,^ 

i^ K chart -Of Fig ia 

it be assun,ed that the . ' 

*- '^e vindow is tn ^h* 

cycle, tl™. , - ne« elocK 

Into a pixel averager transferred 

» as indicated r- 
^t the begteni^g of th ^' 
(tl^e 4, Fig. 14 J _ / f the next oloolc cycle 

- .e ^^^^^^ - 

r. — > one CO J :rrr^ "^^"'"^ 
« indicated by the dashed . 1 '° 

- the be,.n.,„, ^ ^^S. UCB>. 

t; (Fig 14> ^ >=l°ck cycle 

2 ^8. 14). the average value for el 

transferred fro™ the output of th 

f^ls. 13) i„to the elel "'"'^^ 
/ N ^lenient brieh^n« 

At the beginning o^ quantiser 

- P-ls in the dashed ""^ 
. into the. averager e^ y- transferred 
the vindov „e™ory co^tainTth!' 

-1-late element value / ^7 ''^"^ " 

the dashed s^ua^i^;:^;!^^^^'-- 
one places t-h« - ^" 12 (c) t-f 

the ,uanti^ed element values int 

^^es into a shift 
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register delay line memory SR^ (Fig. 13), with 
N-1 taps separated by C/N locations i one can simul- 
taneously obtain element values E^^^ through E^jj. 
To obtain elements in the second through the N^^ 
row (£2^ through Ej^ in Fig. 12(A)), it is also- 
necessary to delay values in the vertical direction 
(I , , etc,;v. After a delay equal in length to j 
one CCD line, the contents of the window will be | 
that indicated at by Fig. 13(d). After a ' 
vertical delay equal to C/N CCD lines, the window 
memory will contain those pixels required to calc- 
ulate element £2^, as shown at 1^^ in Fig, 12(E), 
If one takes the output after C/N vertical CCD line 
delays and places it into a tapped delay line of 
the type described previously, shown at SR" in. i 
Fig. 15, with taps for each C/N pixel location, \ 
one can obtain values for £2^ through £2^^. | 

Expanding this architecture to provide I 

I. 

all values simultaneously for elements E^^^ through t 

I 

^NN* y^®^<^s general configuration shown in Fig. 16. | 

I 
'f 
% 
t ■ 
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Ihe sreate,t feature of the architecture 
is that it provides a „l„ao„ that .oves 1„ one 
Pixel increments in both horizontal and vertical 
■directions over the entire object. Ihis ensures 

that all possible patterns of =i« 

■-erns Of elements (POE's) are 

generated. 



To implement a system that simultaneously 
views the object at different effective .agnifi. 
cations and fields of view, one must calculate 
elements of different size and delay these elements, 
as shown in Fig. 15, for each A,rr 

^ . ^^"^ different masnification 

type. A general structure ch^■^ ™„ 

„„ . , provides element sizes 

ranging from one pixel per element to any number of 
Pixels per element l„ which values for all the differ- 
ent element sizes are provided simultaneously. m.y 
now be described with the aid of the elementary dia- 
Sram of rig. „. computational unit 

«agram of Fig. x8. Considering first the three 

element sizes shown in Fte 17 ~ , - 

, , " '"-e. 17, mask 1,. represents 

a-x2 group of Pixels , 1,. . 3 x 3 group, and 
l3' - 4 - 4 group. If one has already computed 



the sum of pixel brightnesses within the 2x2 

group I^, and now wishes to compute this sura for 

the 3x3 group one only needs to compute the 

partial sum of pixel brightnesses contained within 

band 3 of Fig, 17, and add this partial sura to that 

computed for the 2x2 group I^, Likewise, once 

the sum- for the 3x3 group I2 has been computed, 

one can compute the partial sum for band #4, add 

it to the sura for the 3x3 group I2 and obtain the 

sum for the 4x4 group I^. This process can be 

continued indefinitely. Such, addition (represented 

by summing symbols £.) is employed in the example of 

Fig, 18 later discussed. 

For the system of the invention, one 

would first choose the maximum element size, say 

pixels. The partial summing procedure would then be 

incorporated to obtain all element sizes ranging 
2 

from 1 to M pixels as represented in Fig. 18. 
Alternative structures in which the elements are 
centered one within the other are shown schematically 
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Fl8. 19, and oth«r sl.tUr configurations can 
^Iso be implemented tn similar fashion. For the 
structure sho™ i„ n,. 19, one would compute the 

7 ''''''' -tansular ri„s instead 

of each "1" shaped band. 

Returnins to the example of Fig. 18, the 
computational time of the pi.el averaging „at„or. ■ 

IS shown as a function of the tot^i 

^ ^"''^^ of stages 

M, the number of bits per pixel, and the number of 
b^ts per quantized element, all of which can be 
optimized for a eiven ^,nr,n . 

Inb Stven application. A single pi;„i 

- band *l is quantized to yield a result of one 
Pixel in the element, all three pixels in band «, 
divided b. (.IV) and quantized (g), ,2, 

so on. with all (2M-1) pixels in band #M. 
divided (BIV) b. .2 ^^^^^^^^ ^^^^ ^ 

Pixels in the element. I„ addition, for certain 
applications, the stage of division (.xv) preceding 
the element quantizer (q) in ^ ^ 

vy/ in fig, 18 can be elimln^t-d 
decreasing the computation ti^e. ' 



In the printed circuit card inspection 
example, as an illustration, it is feasible to 
quantize both the pixel and element brightness 
values into two levels indicating the presence 
and absence of metal, as previously described. 
For this application, M is chosen equal to 8, 
yielding elements ranging from 1 to 64 pixels in 
size. The computation time for all eight element 
values from the time the pixels are clocked into 
the 64 pixel window is 200 nanoseconds (ns). The 
sum of pixels brightnesses in each band of the 
window (see Figs, 17 and 18) is calculated using 
high speed. Look-up tables are formed (Fig. 20) 
in the form of programmable-read-only-memories 
(promts), now more fully discussed. In this appli- 
cation, each pixel brightness is represented by a 
logical "1" repriesenting the presence of metal or 
a logic "0" representing the absence of metal, such 
that the sum of pixel brightnesses in a band equals the 
number of pixels in the band equal to a "1". This sum 



is computed by using the pixels in each band as 
the address to a PROM, Fig. 20. and storing as the 
contents of each address the number of bits equal 
to a "l" in that address » 

Thus, in Fig. 20, the group of four 
pixels 11, 12. 21. 22 (corresponding to I, in 
Fig. 17 and band #2 in Fig. 18). is applied to the 
uppermost PROM (2), with the number of pixels in 
the 2 X 2 group $2 equal to a logic "l" being ^4. 
Band #3 (3 X 3 pixels) is shown associated with the 
PROM (3) and band #4 (4x4 pixels), with PROM (4), 
and so oni with advantage taken of the computation 
of the partial sura of pixel brightness contained 
within band #3 by adding the same' (11, 12. 13, 21. 
22, 23, 31, 32. 33) to the pixel brightness for 
14. 24, 34, 44. 43, 42, 41 at the high speed summing 
network ^3. In this example, (the number of 
pixels equal to a logic "1" in band #4) ^7, 

As a further illustration, if the bit 
pattern in a band is given by OlOll, this equals 
the binary address eleven. The contents of address 
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eleven equal 3. The outputs of these PROMs, which 
equal the pixel suras in each band, are then summed 
together to form the final pixel sum for each ele- 
ment size. This final sura is computed using the 
high speed summing networks £.(Figs. 18^ 20), Each 
element value is then compared, as later explained 
in connection with Fig, 21, to a threshold number 
for the specific element size. If the value exceeds 
the threshold number, the element brightness is 
quantized to a logical "l", indicating the presence 
of metal. If the value equals or falls below thres- 
hold, the element brightness is quantized to a "O" , 
indicating the absence of metal. 

While Fig, 20 demonstrates how the pixel 
■ summing is implemented for all eight element sizes 
ELI through ELS, Fig. 21 demonstrates how the thres- 
holding is accomplished for the successive masks. 
The outputs S2-Sg are shown applied to respective 
comparators COMP 2-8,- to which are respectively 
applied the appropriate threshold signals T2-T8 . 
Each of the eight quantized element values ELI 
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through ELS is applied to an element delay unit 
of the type shown, for example, in Fig. 16. A 
separate delay unit is required for each of the 
eight element sizes. Two parameters are required 
to implement the delay" unit i 

N « total number of elements along one axis 
of the K)V, such that 

.2 

N = total number of elements in the FOV which 
is equal to the- number of elements in 
the POE applied to the recognition 
memory I and 
C/N = total number of pixels along one axis 
of an element. 
The values for C/N are determined by the element sij 
and are listed in Table 1 below, 

TABLE 1 

Element Nsm^ Element SIt-.p c/N 



EL 1 
EL 2 
EL 3 
EL 4 
EL 5 
EL 6 
EL 7 
EL 8 



1 pixel 
4 pixels 
9 pixels 
16 pixels 
25 pixels 
36 pixels 
49 pixels 
64 pixels 



1 
2 
3 
4 
5 
6 
7 
8 



2 

The value for N , which is the number of elements 
in the POE, also equals the number of bits applied 
to the address lines of the recognition memories 
("k*, Fig. ll)f since each element value is repre- 
sented by one bit in this particular application. 

2 

The value chosen for N is preferably selected to 

minimize cost and hardware complexity. Specif ically^ 

2 

if- there exists N elements t each represented by 
two possible values ("0" or "l*')t then there exists 
a total of 2 possible patterns. This implies 
that the recognition memory (Fig, 11) must contain 
2 memory locations to store mask information about 
each pattern. The total number of integrated cir- 
cuits (IC's) required to store this information is 
given by the equationi 

nu-ber of ICa - ^^Itll^^^'^^ . 

For this application, high speed static memories 
with read and write times on the order of 100 ns 
were required. The densest of such memories currently 
available contain 16,384 (2*^^) locations with one data 
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btt per location. The total number of these IC's 
required to store 2^ patterns with one bit of 

information used to describe each pattern is given 
by I 

number of IC's = ~ I 

- 2^"^ ' . . I 

. Table 2 lists the number of lC«s required as a f 
function of the number of elements (N^) m the POE. I 

TABLE 2 

^ (9> 5.12 X Iq2 

* (16) 6.SS X 10* 

5^ (25) 3.35 ^ ,(,7 . 

J • • 2,048 $ 

^ (36) 6 87 3f inlO I 

From Table 2 it is obvious that a' 4 x 4 group of elements I 

forming a total ofUbits in the POE (i.e. n2 = \e) 13 J: 

the maximum number of bits that is reasonable to im- ' 
Ple^enz. especially since a separate recognition 

inemory is incorporated for each of the eight element ' ^ 

sizes. In many applications, such as the one tmple- ^ 

mented herein, it is desirable to store more than t 
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one bit of information to characterize a given 
POE. Should this be the case, one can simply 
parallel memory chips. If, for example, numbers 
0 through 7 (3 bits) need to be stored at each of 
65,536 locations, this can be achieved by using 
12 such memory chips. 

It has been discovered that this learning 
process may be vastly improved in speed by proceeding 
in a way that prior communication or similar pro- 
cessing theory, predicated 'on reducing noise, has 
contra- indicated. Specifically, by deliberately 
introducing noise modeled to correspond to actual 
jitter or vibration of the inspection apparatus over' 
time, or actual illumination variations over time, 
the learning time can be significantly reduced." 
"Noise" perturbates the relationship between the 
true signal and particular quantization levels 
that the signal lies between. To" model such 
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noise, we can either perturbate (or vary) the signal 
relative to the threshold (or quantization) levels, 
or perturbate the quantization levels relative to 
the signal. In this embodiment, the system uses 
one quantization level to create binary images 
C> threshold are white, <=threshold are black), and 
the threshold is perturbated relative to the signal. 

Referring to Fig. 22, for example, sub- " 
sonic or low frequency noise modeled in spectrum and 
amplitude to be equivalent to vertical vibration 
noise of the inspection platform in actual operation 
over time, is introduced by adding the same to the • 
analog image signals from the CCD. causing the same 
to be learned and thus recognized when detected, 
rather than eliminated. Large sample sets to be 
learned by standard practice is tin,e consuming.. By 
using the mask samples and modeling noise to corres- 
pond to vibration or variation in illumination (causing 
thicker or thinner lines in the imaging), and super- 
imposing such noise at a rapid rate, on the analog 
image signals before being quantized to digital form 
the machine may be taught large quantities of in- 
formation rapidly. 
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While this concept of thresholding noise 
introduction for teaching mask shifting or vibration 
and illumination variation effects is of more general 
applicability in analog signal-to-digital conversion 
than the apparatus of the invention, it is particu- 
larly useful herein. Noise in the range of .003 hz 
to.l mhz has been so used, providing the following 

rather startling results. 

In an exemplary system, the following fre- 
quency bands of noise were selected to effect graphic 
TV display, simulating the shifting of the object 
being examined (such as the printed circuit cards) 
either due to spatial offsets or vibration or possibly 
variations in illumination. Considering one horizontal 
bit, which in the example . illustrated corresponds to 
a 2 megahertz data rate, the horizontal bands of 
noise frequencies were selected to affect one hori- 
zontal element at each of the eight different mask 
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slzes P1-P8, that is. a single horizontal element 

Change value without afrecting its neighboring ei 

-nts. Por the smallest u sin«.e pi.el). ea,h" 

element, 1 megahertz; for P2 (2^) , 500 kilohertz, 
for P3, 333 kilohertz, for P4, 250 kiloherta? for 
P5. 200 kilohertz; for P6, 166 kilohertzj for P7, 
140 kilohertzf and P8, 125 kilohertz. As for noise 
bands to satisfy vertical direction variations, PI, 
1 kilohertz; P2, 500 hertz; P3, 333 hertz; P4, 250 * 
hertz; P5, 200 hertz, P6, 166 hertz, P7. 140 hertz; 
P8, 125 hertz. 

To vibrate the entire mask as a whole, as 
distinguished from just vibrating each element, as 
above, noise frequencies may be selected as follows. 
.PI, 250 hertz, P2, 125 hertz; P3, 83 hertz; P4, 62 
hertz; P5, 50 hertz; P6, 41 hertz, P7, 35 hertz; 
P8. 31 hertz. Effects such as slow daily variations 
in illuminations which would cause corresponding 
variations in camera signals, may be modeled by 
varying the threshold at a low noise frequency band 
in the range of .003 hertz to 15 hertz. This noise 

also simulates slow frpnnon^., 

irequency wooble or possibly 

shaking Of the entire machine and table thereof. 
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A further refinement that avoids the | 
possibility of the apparatus mistaking an indicated ' | 

abnormality for a defect or other such- pattern of I 
interest, as may be caused by variations about the I 
thresholds, is the use of adjacent location mask | 
outputs to enable the conclusion of a real defect, | 
such as by roonitorins horizontal and vertical contig-' | 
uous masks. More generally, the avoidance of mask 
edge position uncertainty or error in the image of ! 
the successive samples being monitored during the 
line scanning of the object or pattern thereof, with 
resulting pattern uncertainty (including whetJier 
the defect or other sub-pattern of interest is actually 
there), is effected by monitoring adjacent or con- 
tiguous patterns in two dimensions- (horizontally i« 
and vertically, for example), to verify that the 
same sub-pattern defect actually occurs in all ad- 
jacent masks. From one viewpoint, this may be con- I 
sidered also as effectively "vibrating" the mask to I 
avoid uncertainty of image sampling. In practice. f 



This may be effected by storing m succession 
the sets of bits representing horizontally 
adjacent pattern information in a -scanned row or 
line of the image, as before explained. Pig. 23C, 
and comparing the same with similar pattern 
information of a vertically displaced adjacent 
scanned row line Pig. 23E; such that only if all 
pairs of pattern information sets of bits show 
the presence of the sub-pattern, is indication of 
the detection of the defect or other sub-pattern 
indicated. 

As previously described. „,ask5 are com- 
posed Of elements, and elements are composed of 
groups Of pixels ranslng from 1 pixel in size 
many pixels. A pixel's threshold Is set, and if 
the number of white pixels contained within an ele- 
ment group exceeds the minimum threshold, the ele- 
ment is said to be White, else it is black. r„ the 
larger elements vhlch contain many pixels. th.„s- 
holds may be set. for example, such that if r..ce 
than 50^ Of the pixels are white, the element 
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is white. Elements lying on edge boundaries r however t 
may oscillate in value because there exists a certain 
amount of quantization error or noise in the system; 
Following inspection during the learning phase, where 
an element may have been determined to be white be- 
cause 50% of the pixels showed white » should 1 pixel 
change its value during the .run phase, a new pattern 
may be generated, not previously seen during the learn 
mode; and hence this unknown or foreign pattern will 
be flagged as a "defect". Such variations due to 
quantization of edge elements or noise can lead to 
false error detection whichj in accordance with a 
feature of the invention, is obviated by effectively 
"vibrating" the mask in spatial coordinates , both 
horizontally and vertically, at least 1 pixel. If 
a true defect is to be detected as such, the foreign 
or unknown pattern must be detected within plus or 
minus a predetermined -number of pixels in both hori- 
zontal and vertical directions. In other words, the 
defect must be present at all "vibrational" positions 
of the mask; otherwise, it is concluded that the defect 
signal resulted from such quantization and should be 
ignored as such as not constituting a true defect. 
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-partus rro. p.,„ p.e-p.os....,„, 

:r """"" — • -.U„, 
s Xf- o. au.o-Xaa„l„„ object, .o .e 

learned" are placed underneath th= 

u=rneara the system whtch 
proceeds to scan the™, learning all i™ 
rures thereof as the c , ^"P""--: fea- 

f as the complete learning process. No 

rr°"^"' Whatsoever Is needed. Ihe Invention 

""""" ^'^^ - — -isn- 

and operates independently o. sU.ht rotatl 

or displacement of the object Tn 

"ujecr. In addition, the 
nvent o„ ^^^^^^ 

w-Xe ..learning... thus to teach the system al 
^dependent. Specifically, one s.o.la he aW . 
the rotation hy 3soo - "-^ 

'"e Pixel elements, without a 

• ""^^""^ ^"lation. ■ Throuoh 

learning the noise-modeling and eff, 

techniques, moreover the ! 

additionally becomes 

-l»cst completely .ne«ected by any verti , 

Shifts Whatsoever. ""^ horizontal 
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While the specific illustrative embodiments 
of the invention have been described with reference 
to elements composed of contiguous elements, such need 
not be contiguous. The elements may be composed of 
every n data points or other sub- samples of data 
points, or sub-sample groups of data points (for 
example, groups of four pixels at a time, with 
each group spaced by ten or twenty pixels apart). 
This would model in accordance with a kind of averag- 
ing or sub- sampling scheme. It should further be 
noted that the mask does not have to be of square 
configuration as illustrated. It could be triangular 
or round or circular or of any other kind of two- 
dimensional shape whatsoever, so long as the masks 
can be overlapping such that all areas of the object 
are seen by the masks ,. independently of the mask shape. 

An additional feature of the invention is 
that since the same processes all data as it is being 
scanned, with complete information and decision-making 
performed as the data is coming in, there is no 
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necesslty for storing ^h= • 

oring the picture whatsoever t>, 

i«Pllca«„„ or thl, feature u that th 

^ooK at e«re.e^, Ur«e «e:.... J 
-«nl» nez.s. „.thout Xl..tatJ/ '"^^ 
«>e s..te™ can loc. at a row of pr^: J " 
"-paper co.U. out thousa J all""^ 

™Us Of metal ». f - inspect 

"etal, as for pin holes or orh» 
With no u„i,,«„„ °^ -"facts. 

-raste. with prior s.st:„s":t" " """^^ 

- store, storage s. irjlf ^ 

- - .a.e. to astrono.lcal valuT L ^""^ 
cations as rh.. ^""""^ appli- 

newspaper or the like k ^ 

T„ ' ^^^o^e mentioned 

In summary, moreover th. 

-<ierlyln, two modes or i.ol ^^^'^^^^^^^ 

u«s or implementation of^ »-u • 
Won are as follows. 

i„f„ , """^"8 ""ed scan 

information on a kno™ Object sc 

Objects f:n V, . J*"-- similar 

shl " ^"^^^"^^ — ^nes If anv 

.esi;: P-iousl. seen Is present, an., if 

- « in.icatlns the 
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location, for example, of defects or shape varia- 
tions not present in "good" object images • 

In a second mode of operation, the system 
is "taught", as before explained, to differentiate 
between different types of objects by storing scan 
information as to these different objects with 
information for uniquely defining the different 
objects, and then, when inspecting objects, as 
in a bin or on a conveyor belt, identifying the 
same and, if desired, stopping and/or tagging 
the identified objects; 

These modes can, of course, be expanded 
into a very general concept of character recognition 
by causing the machine to scan the various letters 
of the alphabet as different objects and storing 
and tagging the scan information of the different 
letters for totally unique characteristics. In the 
English alphabet, for example, there are 26 characters, 
such that each of the characters may be associated 
with respective tags 1 through 26, Of course, be- 
cause the technique "looks" at general shapes. 
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there is nothing that limits the characteristic 
process to any one alphabet or language or sjanbols. 

While sometimes inspection is required 
at only one magnification, at other times the 
before- described repetition of scanning at different 
magnifications is essential. The concept underlying 
the invention is to provide a- set of magnifications, 
the smallest being chosen, as previously explained, 
to see the smallest object of interest such as a 
defect or. other shape, and to use the largest magni- 
fication which will still show perhaps the largest 
object and still enable identification of that ob- 
ject. By storing pattern information at the lower 
magnification as well as the higher magnification, 
patterns can be recognized at very fast rates, say 
200 nanosecond- type speeds or less. This will re- 
duce and perhaps in some applications eliminate 
the need to reconstruct the larger shapes or peri- 
meters from the smallest 'shapes shown by the 
highest magnifications . 
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The invention thus distinguishes also 



from some other approaches that store specific 
"learned" images and cannot work with new images 
not seen before. In accordance with the invention, 
to the contrary! the system learns and stores in 
recognition memory graphical characteristics which 
it can then identify when such appear even in totally 
new- graphics and images that have not been shown to 
the machine before. Of course, graphical character- 
istics are not the only information that may be 
classified in accordance with , the underlying concepts 
of the invention I the same being applicable to other 
information including other electromagnetic or acoustic 
signal information* if desired. 

Referring to the system of Fig. 22, illustra- 
tive of a practical embodiment of the invention, an 
object is in motion (as upon a moving table or plat- 
form or the like) past the inspection station in the 
upper left, illuminated so that its image is focused 
by a lens upon a linear CCD or other sensing camera 
or the like, with the object motion perpendicular to 
the CCD. The output of the CCD is an analog signal 
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'"ffer memory BM, each V '=™-<il"'ens£onal 

being equal to the len^.^ "'""^^ 

"® length of the Crn 
that a large two ^^^^^ 
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-ional .nit ,3. an ^^T^^^ "^^^ 

created, .e .^.^ZZ J ' 

*e element delay .nit 
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which computes these n,askf was earlier described 
in connection with Fig. I6, a„d the output of each 
of the element delay units is fed to a recognition 
memory. For each different size, there is a unique 
element delay unit that creates the mask composed 
of such elements, referred to as POE (pattern of 
elements), and such are traced onto the address lines 
of the recognition memory, representing a unique 
address or unique pattern. 

During the learning mode of the system, 
an object is shown to the machine and scanned, develop, 
ing these patterns of elements or addresses applied 
to the recognition memory, such that information 
about these patterns is stored as associated with 
the specific object being scanned. In the run mode ■ 
of the apparatus, inspecting an object under test 
and determining whether that object has been seen ' 
before and "learned", the patterns of elements or 
addresses of the object under inspection that 
are developed are compared with the contents of the' 
recognition memories to determine whether these 
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pacterns have been seen before. Ihe analy.l. for 
ac«.al interpretation and correlation of these patterns 
is performed in a unit described as a computer or 
Processor for interpreting and correlating pattern 
mfonnation, performing the foUovins operations. 
First, if any pattern is developed at any of the 
different recognition memories that has not been pre- 
viously seen or learned and addressed therein, this 
constitutes an error or defect or pattern outside 
the norm, and is flagged. For this to be definitely 
identified as a true defect, it may be required 

as before explained, that multiple reco«nltlnn ' 

*^^cognition memories 

all Indicate a foreien oai-i-=~ • 

reign pattern is present at more than 

one Of the so-called view magnifications; or, if only 
certain recognition memories for the given applica- 
tion are Important, that a certain limited number of 
foreign patterns would constitute a defect, m another 
-de, as before outlined, due to pi.el <,uantl.atlon 
errors, a single indication of a defect at a sln.1. 
location on the object Is not sufficient to deter- 
".me that the defect is actually present, but a chec. 
i. effected around a given region of the object, as ■ 
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by requlrlrns at least two contiguous horizontal and 
vertical loications all to flag the presence of the 
defect. Th-.us the output? of these recognition memories 
are viewed .^continuously in real time as the object is 
being scannted, with the information as to whether 
these patteirns have been previously seen or not dur- 
ing the leaiming mode, being continuously monitored 
by the processing and interpretation and correlation 
unit, flaggiing errors if present. It is important 
to note that: the architecture of this technique per- 
mits the peirforming of these correlations at very 
high speeds and with relative simplicity, 

Sunnmarizing the operation of Fig. 22, thus, 
the optical scanning of an area of a region of in- 
terest of th« object is performed by the CCD, Digitiz- 
ing the scanried signals in real time to produce 
successive trains of digital signals corresponding to 
the optical scanned lines, is performed in the pixel 
quantizer. Delaying the successive trains of digital 
signals to produce a vertical raster of successive 
trains of digital signals corresponding to previously 
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optical scanned lines i<: o^r 

V effected in the two-din,en- 

SLonal buffer memory, more fMii 

connexion r , T I """"""^ "™ 

and vertical V pixels, earlier- ^ 

^ earlier described in detail 

in Pigs. 6, 12 and 13 cno. 

Groups Of pixels are 

less than V ana hi u eoual to 

the subscript re " "'^ «' *ete 

scr.pt , represents specific elements rangW 

from an element one Pixel in 3i^e to , v 

« Pixels in Size ref ^° » element V by 

n size, reference being made to the »•> 
computational unit described in rigs 18 Z 
that unit enables select- ^"^ 
for . '"'"^ ^" ^l^»-nt threshold value 

Z : -determining a minimum 

and maximum number of pixel o w 

pixels, such tha^ i' f i-v, 
of ni^^Tr, ^ ^ . ^f '^"^ number 

on pixels contained in -, 

™um and maximum valu I ^ """" 

element value is set ro 

-ththe s.zeof the element, as detailed i„ n^. n. 
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The element delay unit (more fully described in connec- 
tion with Fig. 16) delays each element by Nr 2N, 3N 
etc. horizontal pixels and by N, 2N, 3N, etc. vertical 
pixels to generate a mask representing a visual pattern 
wherein each element of the mask is displayed a full 
element from the previous or contiguous horizontal 
or vertical element. By optically scanning a known 
object t a mask is generated as above to serve as an 
address and a code number Is stored In the' recogni- 
tion memory of Fig. 22, such address indicating the 
teaching of the machine to recognize a desired object 
or pattern. When operating in a run mode with an ob- 
ject to be inspected, the code number* is read to deter- 
mine if the mask generated during the optical scanning 
of the object to be inspected corresponds thereto, 
indicating the recognition of the desired object 
pattern or the lack of correspondence thereto in the 
computer or processor for interpreting and correlat- 
ing pattern information (Fig. 22). If the object or 
pattern has not been previously seen, such is indicated 
or flagged. 
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unxts « the computer o. p„«3.or above- 
referenced and suitable Indicating or dl. , 

--sno„.tc.,.,3.,j;:r.~ 

». ....... ^. jxr: r «- 

present in an Object under inspection a, • 
-sisned sue. that an u„.no. pattern ^^^"'^ 

-Ututes a defect in the Object, 
attained by providing that the ^ , 

recocnltloo " N 

memories is logically ATOed'-^ith an k 
8«e, to determine if that 

and " looked at- 

-rr: 

-LnOLcatLon can be 
presented to the operator by „av of 

ent types of dl.m ^ """"^^'^ °' 

ypes of display devices, including the TV • 

" CFig. 22). „herein the specifi ■ ™- 
- Of the invention is al T"^ 

*at it detects a defect ca^ b 7 " 

" can be frozen and displayed. 
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Thus in the system of Fig. 23 A data from recognition 
memories 1-N are applied to respective gates receiving 
enabling signals from masks 1-N and provide an output 
A, such as a logic 1, representing a defect indicative 



23B/in5ures that if defects exist simultaneously at 
chosen masJcs, then and only then is it considered 
that a defect actually exists in the object, as before 
explained. There is effected through the application 
of the desired data from the memory to respective 
EX/OR and AND gates, again enabled as in Fig. 23A, 
providing an output B in a final AND gate as in- 
dicated. Fig. 23C shows the application of output A 
of Pig. 23A or output B of Fig. 23B to an AND gate 

also inputted from successive unit horizontal delay 
elements l-N. Again in this system, a logic 1 output 
would indicate a defect from the previous section? 
and if and only if a defect is detected at contiguous 
horizontal locations in the same horizontal line, would 
the machine indicate the actual presence of a defect 
in the object.' This circuit thus enables the contiguous 



of the fact that an unknown pattern has appeared in 
at least one of the memories. The circuitry of Pig. . 
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chec. or before described. A logic 1 1„ «,e 

output C would thus indicate that a defect vas 
na^ed b. one or .ore recognition .e^orles at each 
Of the previous N horizontal positions.. 

Similar circuitry for achieving the con- 
t^suous vertical „as. defect verification before 
---bed IS She. m . 

■produce output D. If only if , . 
at M ^« detected 

at „ eontlsuous vertical locations at the sa„e ho.l.on 

tal location. does the circuit indicate the presence 

Of a defect in the object Th„« , " 

^' ^us a logic 1 in the 
output D indicates that a defect was fla..eH . 
or more reco^iitinr, ^^SSed by one 

recognition memories at each of fch. ^ . 

• ''he previous 

M vertical lines at- = « 

lines at a conunon horizontal position. 

Lastly, the circuit of Pi^, p^F ».v,. 
, ^^e- 23E achieves 

the "vibration" of the mask tn ^ 

mask to insure indication of a 
genuine defect as orAv-r ...g-, „ ^ , , 

fche h - ^ e^PJ-alned. This combines 

the horizontal and vertical nr. 

°°"^isuous mask checking 
-or defects with the outputs air 

P"^^ ^-J ^:^om successive verti- 
cal line delays, the outputs b v r 

Inputs b-k from unit horizontal 
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delays, and the outputs c-Lfrom the Nth horizontal 
delays, operating the final AND gate such that a logic 
1 will indicate that a defect is present if and only 
if all previous N horizontal locations at each of 
the previous M vertical lines indicate a defect. 

An operating system of the type shown in 
Fig. 22 had the following technical specifications 
and 'specific parts and equipment. The motor drive 
for an XY table (of Ralmike Corporation) was con- 
trolled by stepping motors (from Sigma Corporation}. 
The imaging lens was a Photar enlarging lens by 
Leitz (I30mra), and the CCD camera was constructed 
by Beltronics of Brookline, Massachusetts, using a 
Reticon 1728 element array. The clock drivers for 
the CCD were DSOO 26 National Company chips and the 
linear amplifiers and sampling-hold circuitry utilized 
LH0023 chips. The noise generators were: an H. H. Scott 
generator producing noise from 20 cycles to the 1 mega- 
hertz range, another generator that produced a pink noise 
spectrum from 20 hertz to 16 kilohertz, and a third genera- 
tor by Wavetek producing noise from 0.003 hertz to 20 hertz 
The pixel quantizer in this application was a single 
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LM361 comparator. The trv.« ^< 
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computed a sum of pixels, such is compared to its 
pixel threshold, with the comparison being performed 
using 74S85's. The element values were then sent " 
into the individual element delay vmits constructed 
using 7^S299's, 7^LS37^'s, 7^l6i|'s, and Hitachi • 
TMM20l6»s. These element delay units were used to 
compute the mask addresses, which are applied to the 
recognition memories. The memories were implemented 
using Hitachi HM6l67'5 memory chips. The computer 
or processor for interpreting and correlating pat- 
tern information was implemented using 7^s86's to 
perform the exclusive OR function described in Pig. 
23, in addition to the 7.^S6^'s and a variety of 
flip-flops. The delay lines were of the type ThIM20l6 
to implement the various vertical delays necessary 
for some of the defect detection shown in Fig. 23 j 
and again 74164 's were used to implement some of the 
horizontal delays. Additional control logic contained 
7^Sl63's and 7^IS299's for enabling "and disabling the 
appropriate masks at the correct time. 

The overall systems specifications were 
as follows. For demonstration purposes, the XY table 
was capable of scanning an area 8 inches by 5 inches. 
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Larger tables, however, that can easily scan tvo- 
foot squared areas and larger, can readily be hooked 
into the system. The resolution of the CCD system 
was of the order of 1 mil. rne processing or data 
rate vas from 5 megahertz (200 nanoseconds) to as 
low as 2 megahertz (500 nanoseconds). Operating 
the CCD at a 5 megahertz rate, the time required to 
clock out the complete array was half a millisecond, 
so that the time required to analyze a strip 8- long 
by 1" was 5 seconds. The time required to scan a 
PC card 8" by 10" was 50 seconds at IX optical magni- 
fication. If operation is effected at a magnification 
of 1/2 X, then the complete time required for analysis 
of the 10" by 8" card would be reduced to- 12 seconds. 
It should be noted here that the only limitation in 
operating at 1/2 X or IX is the quality of the optical 
lens and not in any way the inspection system. In 
addition, these processing times of 50 seconds or 12 
seconds are the times required with only one CCD 
camera and one complete analysis system. Clearly, 
such analysis systems can be cascaded by having 
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multiple CCD's looking at the image such that the 
various sections of the image are overlapped. If 
this is done, there is actually no limit to the 
processing time. For example, even for IX magnifi- 
cation where one unit would take 50 seconds, two 
units would take 25 seconds, five units would take 

10 seconds, and the processing time is linearly pro- 
portional to the number of systems viewing the object. 

As before discussed, where desired, the TV 
or other monitor M of Fig. 22. may have its imagp of 
the pattern of interest of 'the object, such as a 
defect, frozen on the screen with the aid of the TV 
memory buffer, so labelled, and with x-y or other 
coordinates or position of the same also recorded, as 
indicated. 

It is to be understood that inspection 
apparatus and the like embodying the various features 
of the invention may, if desired, be used without 
all of the features incorporated therein, and that 
further modifications will occur to those skilled 
in this art, and such are considered to fall within 
the spirit and scope of the invention as defined 
in the appended claims. 
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What Is claimed isi " 
A method of real-time inspection of objects 
at a predetermined region, that comprises, 
optically scanning successive lines of an area 
Of interest of the object at said region; digiti- 
zing the scanned signals in real time to produce 
successive trains of digital signals corres- 
ponding to the optical scanned lines, delay- 
ing the successive trains of digital signals 
to produce a. vertical raster of successive 
trains of digital signals corresponding to 
previously optically scanned lines, reducing ■ 
the vertical raster to a sub-raster contain- 
ing a predetermined number of substantially 
horizontal (H) and vertical (V) pixels, 
extracting from the. sub-raster group of 
pixels h by V in spatial distribution such 
that v.^V and h^^H. where the subscript i 
represents the specific elements ranging 
from an element one pixel In size to an 
element V by H in size, selecting an element 
threshold value for each element size by 
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predeterinlning a mlnlmuin and maximum number 
of pixels such that if the number of pixels 
contained in an element lies within said 
minimum and maximum thresholds the element 



thresholds varying with the size of the element; 
delaying each element by N, 2N, 3N, etc. hori- 
zontal pixels and by N, 2N, 3N, etc. vertical 
pixels to generate a mask representing a visual 
pattern, wherein each element of the mask is dis- 
placed a full element from the orevious or 
contiguous horizontal and vertical element; 
in a teaching mode, optically scanning a known 
object to generate a mask as above, serving as an 
address, and storing a code number at each 
address to teach the recognition of the desired 

object; when operating with an object to be 
inspected in run mode, reading the code num- 
ber to determine if the mask generated during 
the optical scanning of the object to be 



value is set to the value 0 or 1 , with such. 
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inspected corresponds thereto indicating 
the recognition of the desired object, or 
does not correspond thereto r and indicating 

if the object has not been previously 

seen. 

A method as claimed in claim 1 and in which 
the reading determines an object not corres- 
ponding to the code number, such as a defect; 
and indicating the presence of the defect. 

A method as claimed in claim 1 and in which the 
element size is varied in effect to vary the 
magnification of inspection of the. object, 

A method as claimed in claim 1 and in which 
optical scanning is effected of a plurality 
of known objects, generating corresponding 
masks serving as addresses, with code numbers 
stored at each address that are distinctive 
or unique for each object. 

A method of real-time high-speed inspection of 
objects with the aid of video pick-up . cameras , 
that comprises, storing digital signal mask 
information corresponding to video Images of 
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regions of predetermined field of view of 
a known object at effectively different 
magnifications! successively scanning contig- 
uous regions of an object with such a camera, for 
each of said predetermined fields of view, 
• and digitizing in binary form the image scans 
to generate digital signal mask information, 
, repeating such scanning at effectively greater 
magnification with the same field of view to 
generate magnified digital signal mask informa- 
tion! comparing the generated digital mask 
information at different magnifications with 
the stored digital mask Information to identify 
known or unknown portions of the object, and 
indicating the identification of such known or 
unknown portions, 
A method as claimed in claim 5 and in which the " 
generating of digital signal mask information 
from said scanning at greater magnification is 
effected by averaging or weighting the digital 
binary data corresponding to picture points of 
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the scanned image, such that a certain 
threshold number of common picture points 
or their corresponding binary units in the 
mask will determine the general white or 
black character of the same. 
A method as claimed in claim 6 and in which 
the stored digital signal mask information 
comprises desired or good object portion 
masks, and said comparing step indicates 
defects or deviations by lack of matching 
the scan-generated digital signal mask in- 
formation with the stored mask information. 
A method as claimed in claim 6 and in which 
the stored digital signal mask information 
comprises predetermined defects or undesired 
deviations, and said comparing step identified 
such in the scan-generated digital signal 
mask information by matching with the stored 
mask information. 



A method as claimed In claim 5 and In which 
the inspection steps are performed in 
parallel by a plurality of pick-up cameras 
scanning substantially overlapping areas 
of said objects. 
A method of real-time high-speed inspection 
of objects with the aid of image sensors and 
the like, that comprises, generating signal 
images of an object with such sensors at a 
plurality of magnifications i reconstructing 
said images into fields of view that employ 
a minimum number of predetermined required 
light levels or colors to distinguish fea- 
tures of such object J dividing each image 
field at each magnification Into discrete 
spatial elements J and. locating patterns of 
Interest In such object by one or more of 
the following I (a) examining the elements 
from each magnification Independently, 
searching for a pattern of Interest within 
each field J (b) correlating patterns of the 
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saine object area generated at different 
magnifications I (c) combining pattern 
information generated from contiguous fields 
at a single magnification, and (d) recogniz- 
ing pattern information combined from contig- 
uous or overlapping fields at different 
magnifications, and correlating the obtained 
pattern information. 
11. A method of real-time high-speed inspection 

of objects with the aid of image sensors and • 
the like, that comprises, generating signal 
images in the form of pixels of an object with 
such sensors at one or more magnifications; relatively 
moving the object in a direction past the sen- 
sor, quantizing the pixels of the. images, into 
a raintraura number of light levels or colors 
required to characterize the object, storing 
t:he quantized pixels to create a two-dimensional ' 
stored image equal to the largest desired field 
of view, selecting predetermined desired fields 
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of view and magnification from said two- 
dimensional stored image j dividing each such 

2 

selected, field of view into N elements j 
setting the brightness value of each element 
substantially. equal to the average brightness 
.value of the pixels contained within the ele- 
ment; quantizing each -such element brightness 
value into a minimum number of threshold levels 
required to recognize patterns of interest; 
applying each quantized pixel group at each 
magnification as a pattern of elements-. to a 
corresponding recognition memory, with each 
pattern of elements serving as an address to 
said memory; entering information into accessed 
memory address locations in order to teach the 
properties of known objects and characteristics 
thereof for subsequent comparative use in the 
method; and thereafter recognizing such pre- 
viously taught information as to objects and 
their characteristics by reading information in 
each memory location accessed by a pattern of 
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elements and using such information to charac- 
terize the object. 

12. A n-ethod as claimed t„ claim li and In which 
said threshcid levels are supplemented by ! 
. the introduction- Of notse correspondlne to 
vibration and illumination variation during 
said inspection, 
13. A method as claimed In claim 11 and In which 
the said recognizing involves verification 
Of two adjacent location images to identify 
the characterization of the object or a part 
thereof. 

M. A method as claimed in claim u and in which 
noise is generated of spectrum and amplitude 
that models the real noise variation that the 
Image sensor system encounters in its actual 

operation Of generating signal images, adding 
such modeled noise to the analog image signals, 
and With said quantizing comprising digitizing 
sampled image signals with added noise to 
enable the digital conversion to recognize 
said signals in all conditions of such actual 
Operation. 
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IS. In an analog-to-digital conversion apparatus 
operating with analog signals from an analog 
signal generating system that from time- to 
time may be subject to noise variations In 
its actual operation, a method that ' comprises 
generating noise of spectrum and amplitude 
that models the real noise variations that 
the analog system may encounter in said 
actual operation, adding such modeled noise' 
to the analog signals generated by said 
system, and sampling said analog signals with 
added noise and quantizing the sampled signals 
to enable the digital conversion to recognize 
said signals in all conditions of such actual 
operation, 

16, A method as claimed in claim 15 and .in which 

the digital signals resulting from said quantizing 
are applied to memory as learned addresses. 

17. A method, as claimed in claim 16 and in which 

the application to memory Is effected over 
time with varying noise models. 
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18. A method as claimed in claim 11 and in which 
said recognizing comprises monitoring the 
characteristics in the patterns of the object 
iinase at adjacent locations in two dimensions 
to search for particular sub-patterns of i^r- 
terest, and indicating the detection of such 
sub-patterns only in the event that the monitor- 
•ing at all of said adjacent two-dimensional 
locations indicates the presence of the" sub- 
pattern , 

19. A method as claimed in claim 18 and in which 

said sub-pattern is a defect or the like, and 
said two-dimensional locations comprise adjacent 
horizontal and adjacent vertical object images. 

20. A method as claimed in claim 18 and in which 

said monitoring comprises storing, in succession, 
sets of bits representing a pair of horizon- 
tally adjacent patterns in a line scan of the 
image, and comparing the same with similar 
pattern information of a vertically displaced 
adjacent scanned line, such that only if all 
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pairs of such pattern information sets of 
bits show the presence of the sub-pattern, 
is such indicating effected, 
A method of verifying the detection of a sub- 
pattern of interest in an image pattern under 
inspection, that comprises, monitoring the 
pattern at adjacent locations extending in 
two dimensions to search for said sub-pattern 
of interest; and indicating such detection 
of the sub- pattern only in the event that the 
monitoring at all of said adjacent two- dimen- 
sion locations indicates the presence of the 
sub- pattern. 
A method as claimed in claim 21 and in which 
said sub-pattern is a defect or the like, and 
said two-dimension locations comprise adjacent 
horizontal and adjacent vertical object images. 



. A method as claimed in claim 21 and in which 

said monitoring comprises storing, in succession, 
sets of bits representing a pair of horizon- 
tally adjacent patterns in a line scan of the 
image} and comparing the same with similar 
pattern information of a vertically displaced 
adjacent scanned line, such that only if all 
pairs of such pattern information sets of. bits 
show the presence of the sub- pattern, is such 
indicating effected, 
A method as claimed in claim 5 and in which 
said indicating is effected by displaying 
said known or unknown portions on a monitor. 
A method as claimed in claim 5 and in which 
the coordinates or location of said indicated 
known or unknown portions is Indicated. 
Apparatus for real-time high-speed inspection 
of objects with the aid of image sensors and 
the like, having, in combination, image 
sensing means for generating signal images in 
the form of pixels of an object at one or 
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mo re magnifications; means for relatively 
moving the object in a direction past the 
sensing means; means for quantizing the 
pixels of the images into a minimum number of 
light levels or colors required to charac- 
terize the object; means for storing the * 
quantized pixels to create a two-dimensional 
stored image equal to the largest desired 
field of 'View; means for selecting predeter- 
mined desired fields of view and magnifica- 
tion from said two-dimensional stored image; 
means for dividing each such selected field 
of view into elements; means for setting 
the brightness value of each element' substan- 
tially equal to the average brightness value 
of the pixels contained within the element; 
means for quantizing each such element 
brightness value into a minimum number of 
threshold levels required to recognize pat- 
terns of interest; means for applying each 
quantized pixel group at each magnification 
as a pattern of elements to a corresponding 
recognition memory, with each pattern of ele- 
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nients serving as an address to said memory; 
means for entering information into memory 
address locations in order to teach the appa- 
ratus properties of known objects and charac- 
teristics thereof; and means for thereafter 
recognizing such previously taught informa- 
tion as to stored objects and their charac-' 
teristics by means for -reading information in 
each memory location accessed by a pattern of 
elements; and means for monitoring such 
information to characterize the object. 
Apparatus as claimed in claim 26 and in which 
said threshold levels are supplemented by the 
Introduction of noise corresponding to at 
least one of vibration and illumination vari- 
ation during said inspection. 
Apparatus as claimed in claim 26 and in which 
the said recognizing means comprises means 
for monitoring the characteristics in the 
patterns of the object image at adjacent 



locations in two dimensions to search for 
particular sub-patterns of interest; and 
means for indicating the detection of such 
sub-patterns only in the event that the moni- 
toring at all of said adjacent two-dimen- 
sional locations indicates the presence of 
the sub-pattern* 
Apparatus as claimed in claim 26 and which 
means is provided resporis-ive to deviations 
from previously- taught and/or stored 
information to indicate a non-conforming 
object or part thereof such as a defect and 
the like. 

Apparatus as claimed in claim 29 and in which 
means is provided for displaying the non-con- 
forming object or part thereof. 

Apparatus as claimed in claim 29 and in which 
means" is provided for indicating the coordi- 
nates or location of the non-conforming ob- 
ject or part thereof. 
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32. Apparatus as clain,ed i„ claim . 

26 and in which 
»e sensing „eans comprises a pa^allt. „f 
Plcic-up catenas To. scanning substantially 
overlapping areas or objects. 

33. Apparatus for re»i 

real-tl^e high-speed inspection 
of objects with the aid of i»= 

"° i^age sensors and 
the like, having. i„ combination, i„age 

sensing means for generating sl^al Images 

Pt an object a-f- o «t 

i^jbct; at a plurality of 

magnifications: means rn,. 

, ineans for reconstructing said 

images into fields of vie. that employ a • 
-inimu™ number of predetermined required 

features of such object; means fo. dividing 
each image field at each magnification into 

N discrete spatial elements- ' 

->.cjueni;s, and means for 

locating patterns of inter.., . 

^"^erest in such object 

°- - Of the following: (a) means 

for examining the elements from each 
"■agnlficatlon Independently to ioc»t= a 
pattern of interest within each field; (b) 

means for correlafcin* 

rreiating patterns of the same 

Object area generated at different 
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magnifications; (c) means for combining pat- 
tern information generated from contiguous 
fields at a single magnification; and (d) 
means for recognizing pattern information 
combined from contiguous or overlapping 
fields at different magnifications to corre- 
late the obtained pattern information. 
Apparatus for real-time Kigh-speed inspection 
of objects having, in combination, means for 
scanning the objects; means responsive to 
the scanning for storing digital signal mask 
information of such scanning of- the objects 
at different magnifications but with 
substantially the same field of view; means 
for teaching the apparatus information on 
known or desired objects by scanning the same 
and storing the resulting digital signal mask 
information; and means, for comparing digital 
mask information obtained by run scans of 
objects-to-be-inspected, at said different 
magnifications, with the stored signal mask 
information; and means responsive to said 
comparing. for differentiating the presence of 
desired objects or objects deviating 
therefrom. 
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Apparatus as in 'claim 3^ and in which means is 
provided for rendering the same substantially 
independent of orientation of the object 
presented to the scanning means. 
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36. A method of real-time high speed inspection of | 

I 

objects, that comprises, presenting signals I 
representing images of an object or parts I 
thereof at effectively different magnifica- j 

tions to constant f ield-of-view processors. t 

I 

one corresponding to each magnification and | 

having its own respective memory; storing the | 

f 

signals of the images in. -the respective memo- f 
ries, thereby to learn the shapes in the { 
images; presenting signals representing images 
of objects-to-be-inspected at said effectively 
different magnifications to said constant- 
f ield-of-view corresponding processors; and 
comparing the said stored signals with the 
last-named signals to determine signals repre- 
sentative of shapes not previously learned. t. 
37, A method as claimed in claim 36 and in which 

the signal, images at different magnifications 

of the objects-to-be-inspected are pertubated 

by amounts small compared to the resolution 

element at the corresponding magnification to 

discriminate false errors from shapes not 

learned . 



-106 



A method as claimed i„ claim 36 and in which 
the said presenting at effectively different 
magnifications is effected by providing 
signals in memory corresponding to an image 
having the lowest magnification, but with 
the resolution of the highest magnification; 
and extracting different portions of the 
image for the different magnifications and 
quantizing the same into'sub-elements, the 
sub-elements making up the said constant 
field of view. 
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